Inpainting in Multi-image Stereo

In spite of numerous works on inpainting, there has been little work addressing both image and structure inpainting. In this work, we propose a new method for inpainting both image and depth of a scene using multiple stereo images. The observations contain unwanted artifacts, which can be possibly caused due to sensor/lens damage or occluders. In such a case, all the observations contain missing regions which are stationary with respect to the image coordinate system. We exploit the fact that the information missing in some images may be present in other images due to the motion cue. This includes the correspondence information for depth estimation/inpainting as well as the color information for image inpainting. We establish our approaches in the belief propagation (BP) framework which also uses the segmentation cue for estimation/inpainting of depth maps.